Concurrent validity measures whether the instrument allows for distinctions to be made between groups that should be separately identifiable. Convergent validity asks whether responses to items in the instrument which are conceptually related move in similar ways relative to each other. And divergent validity measures whether operationalized variables in the instrument are separately identifiable from other, unrelated concepts. The threats to construct validity include not clearly defining variables operationally before instrument construction, using a too-narrow concept for measurement or a too-narrow treatment for measurement that does not reflect the full nature of the concepts measured, improperly measuring the effects of multiple treatments related to the variable in ways that do not account for interaction of other variables, failure to account for the effects of such considerations as subject apprehension or testing effects on subject responses, and experimenter bias. The researcher can control for these threats by linking the theoretical rationale for measurement with the actual measurements, conducting multivariate methodological treatments to ensure that variables moved in the expected directions and with the appropriate similar variables, and through pattern matching to show how variables relate to one another, relevant to expectations.
Internal validity consists of an assessment of whether the instrument shows proper causal relationships between variables. Creswell (2009) argues that there are a number of different types of concerns related to internal validity, consisting of such factors as confounding (or spurious) relationships, selection bias or maturation, test-retest effects, instrument changes,...
The idea here is that when control groups are selected, multiple tests are conducted, or instruments are tweaked, the possibility for changes in responses complicates the likelihood that the research design will be shown valid. Researchers can control for internal validity by conducting research according to accepted limits and methods, conducting appropriate statistical procedures to measure and remove bias and related errors, and reporting methodology with transparency.
If properly accounted for, validity concerns need not harmfully impact upon research. However, avoiding threats to validity requires careful attention to research methodology and appropriate application of common sense. The treatment of validity issues requires up-front planning in the research design and time must be built into the planned methodology to account for the application of appropriate controls. These are required, however, if validity concerns are to be managed. If such controls are not put in place validity concerns can ultimately cause critics to reject the findings due to an invalid research design.
Works Cited
Creswell, J. (2009). Research design: qualitative, quantitative, and mixed method approaches, 3rd Edition. London: Sage.
Trochim, W. (2008). Research methods: knowledge base. Retrieved December 19, 2009 from http://www.socialresearchmethods.net/kb/pmconval.php.
Wood, M.S. & Johnsrud, L.K. (2001, November). Post-tenure review: What matters to faculty. Refereed paper presented at the annual meeting of the Association for the Study of Higher Education (ASHE), Richmond, VA.
